{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "pytorch_quick_start.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/omarsar/pytorch_notebooks/blob/master/pytorch_quick_start.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9XHd5ExbUIUg",
        "colab_type": "text"
      },
      "source": [
        "# PyTorch 1.2 Quickstart with Google Colab\n",
        "In this code tutorial we will learn how to quickly train a model to understand some of PyTorch's basic building blocks to train a deep learning model. This notebook is inspired by the [\"Tensorflow 2.0 Quickstart for experts\"](https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/quickstart/advanced.ipynb#scrollTo=DUNzJc4jTj6G) notebook. \n",
        "\n",
        "After completion of this tutorial, you should be able to import data, transform it, and efficiently feed the data in batches to a convolution neural network (CNN) model for image classification.\n",
        "\n",
        "**Author:** [Elvis Saravia](https://twitter.com/omarsar0)\n",
        "\n",
        "**Complete Code Walkthrough:** [Blog post](https://medium.com/dair-ai/pytorch-1-2-quickstart-with-google-colab-6690a30c38d)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KzsiN3l_Vy1p",
        "colab_type": "code",
        "outputId": "a5f58f2e-64e6-41da-ca0f-73801869c8e0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 374
        }
      },
      "source": [
        "!pip3 install torch==1.2.0+cu92 torchvision==0.4.0+cu92 -f https://download.pytorch.org/whl/torch_stable.html"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Looking in links: https://download.pytorch.org/whl/torch_stable.html\n",
            "Collecting torch==1.2.0+cu92\n",
            "\u001b[?25l  Downloading https://download.pytorch.org/whl/cu92/torch-1.2.0%2Bcu92-cp36-cp36m-manylinux1_x86_64.whl (663.1MB)\n",
            "\u001b[K     |████████████████████████████████| 663.1MB 20kB/s \n",
            "\u001b[?25hCollecting torchvision==0.4.0+cu92\n",
            "\u001b[?25l  Downloading https://download.pytorch.org/whl/cu92/torchvision-0.4.0%2Bcu92-cp36-cp36m-manylinux1_x86_64.whl (8.8MB)\n",
            "\u001b[K     |████████████████████████████████| 8.8MB 44.2MB/s \n",
            "\u001b[?25hRequirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from torch==1.2.0+cu92) (1.16.4)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from torchvision==0.4.0+cu92) (1.12.0)\n",
            "Requirement already satisfied: pillow>=4.1.1 in /usr/local/lib/python3.6/dist-packages (from torchvision==0.4.0+cu92) (4.3.0)\n",
            "Requirement already satisfied: olefile in /usr/local/lib/python3.6/dist-packages (from pillow>=4.1.1->torchvision==0.4.0+cu92) (0.46)\n",
            "Installing collected packages: torch, torchvision\n",
            "  Found existing installation: torch 1.1.0\n",
            "    Uninstalling torch-1.1.0:\n",
            "      Successfully uninstalled torch-1.1.0\n",
            "  Found existing installation: torchvision 0.3.0\n",
            "    Uninstalling torchvision-0.3.0:\n",
            "      Successfully uninstalled torchvision-0.3.0\n",
            "Successfully installed torch-1.2.0+cu92 torchvision-0.4.0+cu92\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uF1P_cRoWpvM",
        "colab_type": "text"
      },
      "source": [
        "Note: We will be using the latest stable version of PyTorch so be sure to run the command above to install the latest version of PyTorch, which as the time of this tutorial was 1.2.0. We PyTorch belowing using the `torch` module. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Su0COdCqT2Wk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.nn.functional as F\n",
        "import torchvision\n",
        "import torchvision.transforms as transforms"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rXCYmmjyVRq5",
        "colab_type": "code",
        "outputId": "a9ea67e6-cd29-4c4e-bb8f-127eac9ab764",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "print(torch.__version__)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1.2.0+cu92\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hhuQyU7AYE6K",
        "colab_type": "text"
      },
      "source": [
        "## Import The Data\n",
        "The first step before training the model is to import the data. We will use the [MNIST dataset](http://yann.lecun.com/exdb/mnist/) which is like the Hello World dataset of machine learning. \n",
        "\n",
        "Besides importing the data, we will also do a few more things:\n",
        "- We will tranform the data into tensors using the `transforms` module\n",
        "- We will use `DataLoader` to build convenient data loaders or what are referred to as iterators, which makes it easy to efficiently feed data in batches to deep learning models. \n",
        "- As hinted above, we will also create batches of the data by setting the `batch` parameter inside the data loader. Notice we use batches of `32` in this tutorial but you can change it to `64` if you like. I encourage you to experiment with different batches."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tSjjLXrOVWBy",
        "colab_type": "code",
        "outputId": "47502e82-f178-452b-995f-8a469670a471",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 285
        }
      },
      "source": [
        "BATCH_SIZE = 32\n",
        "\n",
        "## transformations\n",
        "transform = transforms.Compose(\n",
        "    [transforms.ToTensor()])\n",
        "\n",
        "## download and load training dataset\n",
        "trainset = torchvision.datasets.MNIST(root='./data', train=True,\n",
        "                                        download=True, transform=transform)\n",
        "trainloader = torch.utils.data.DataLoader(trainset, batch_size=BATCH_SIZE,\n",
        "                                          shuffle=True, num_workers=2)\n",
        "\n",
        "## download and load testing dataset\n",
        "testset = torchvision.datasets.MNIST(root='./data', train=False,\n",
        "                                       download=True, transform=transform)\n",
        "testloader = torch.utils.data.DataLoader(testset, batch_size=BATCH_SIZE,\n",
        "                                         shuffle=False, num_workers=2)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\r0it [00:00, ?it/s]"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to ./data/MNIST/raw/train-images-idx3-ubyte.gz\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "9920512it [00:02, 3643813.85it/s]                             \n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Extracting ./data/MNIST/raw/train-images-idx3-ubyte.gz to ./data/MNIST/raw\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "\r0it [00:00, ?it/s]"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz to ./data/MNIST/raw/train-labels-idx1-ubyte.gz\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "32768it [00:00, 57582.77it/s]                           \n",
            "0it [00:00, ?it/s]"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Extracting ./data/MNIST/raw/train-labels-idx1-ubyte.gz to ./data/MNIST/raw\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to ./data/MNIST/raw/t10k-images-idx3-ubyte.gz\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "1654784it [00:01, 973571.63it/s]                             \n",
            "0it [00:00, ?it/s]"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Extracting ./data/MNIST/raw/t10k-images-idx3-ubyte.gz to ./data/MNIST/raw\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to ./data/MNIST/raw/t10k-labels-idx1-ubyte.gz\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "8192it [00:00, 21777.08it/s]            "
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Extracting ./data/MNIST/raw/t10k-labels-idx1-ubyte.gz to ./data/MNIST/raw\n",
            "Processing...\n",
            "Done!\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0nZwZukWXUDn",
        "colab_type": "text"
      },
      "source": [
        "## Exploring the Data\n",
        "As a practioner and researcher, I am always spending a bit of time and effort exploring and understanding the dataset. It's fun and this is a good practise to ensure that everything is in order. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NW_loWKga7CH",
        "colab_type": "text"
      },
      "source": [
        "Let's check what the train and test dataset contains. I will use `matplotlib` to print out some of the images from our dataset. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zWd9Pt1Ca6K9",
        "colab_type": "code",
        "outputId": "1c02a3b5-f5bb-4c51-a999-52d0472f43af",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 220
        }
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "\n",
        "## functions to show an image\n",
        "def imshow(img):\n",
        "    #img = img / 2 + 0.5     # unnormalize\n",
        "    npimg = img.numpy()\n",
        "    plt.imshow(np.transpose(npimg, (1, 2, 0)))\n",
        "\n",
        "## get some random training images\n",
        "dataiter = iter(trainloader)\n",
        "images, labels = dataiter.next()\n",
        "\n",
        "## show images\n",
        "imshow(torchvision.utils.make_grid(images))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAADLCAYAAABgQVj0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzsnXd4FFXXwH8DhEACBAmRmtCkSkR6\nbwLShAACIkVUhFcEQUAp6guIwiclKqJSBUFAehGkGEDhlSZNaVKkBwRCCRICQnbP98dmLrtphCS7\nE+L9Pc99sjN7Z+dkdvbMueeee44hImg0Go0m45LJagE0Go1G4160otdoNJoMjlb0Go1Gk8HRil6j\n0WgyOFrRazQaTQZHK3qNRqPJ4LhN0RuG0cwwjKOGYfxpGMZQd51Ho9FoNEljuCOO3jCMzMAxoAkQ\nDuwCXhSRw2l+Mo1Go9Ekibss+mrAnyJyUkTuAguAEDedS6PRaDRJkMVNn1sIOOe0HQ5UT6yzYRh6\nea5Go9E8PFdEJOBBndyl6B+IYRi9gF5WnV+j0WgyAGeS08ldiv48EOi0XTh2n0JEpgHTQFv0Go1G\n407c5aPfBZQ0DKOYYRhZgU7A9246l0aj0WiSwC0WvYjEGIbRF1gPZAZmisghd5xLo9FoNEnjlvDK\nhxZCu240Go0mJewRkSoP6qRXxmo0Gk0GRyt6zQOpU6cO06dPZ/r06dhsNpfWvXt3q8XTaNIlXl5e\n5MyZk5w5c7J+/Xq2bt3K1q1b1W+nffv2tG/f3iOyWBZemdaMHz+esmXLsnbtWgCmTZvGvXv3LJYq\naX7//XcAAgICaNSoEX/88YfFEt0nd+7c1KhRA4AZM2ZQoEABAOx2u0u/d955hyeeeAKA0aNHc+fO\nHc8KGofp06fTpUsXwPGA2rt3r6XyJISPjw8Af/zxB/369WPlypUWS6RxB6+//jqffvqp2jYMAwAr\n3OXaotdoNJoMToax6EWEZs2a0axZMwAKFy7Mn3/+CcCOHTs4dCh9Bf2UKFGC3LlzA5A/f36WL19O\nmTJlLJbKQe7cuVm6dCn16tV7YN/SpUvz7rvvAhAWFsaWLVvcLV6SnD59mmzZsgFQsmTJdGnRe3l5\nARAYGKhGQ5qUMWLECAD27t3LqlWrEu03fPhwPvjgAwCqVq3K7t273SrXyJEj6d+/f4LvRUREMH/+\nfLfL4IKIWN4ASW2rXr26xMTEJNhu374t/fv3l8qVK0vlypVTfa60aF26dBG73a7auXPnLJfJbIsX\nL5Z79+6pduDAAdm4caNs3LhRmjRpIv369ZOoqCiJioqSe/fuic1mE5vNJvXq1bNc9m7duqlr+sMP\nP1guT0LNz89P/Pz8xG63S9u2bS2XJzktICBApkyZIps3b5bNmzeL3W4Xm80mly9flsuXL0vXrl0t\nkatv377St29fmT9/fqJ9ihcvLtHR0eq+qFKlitvkyZUrl+TKlUsOHz7sooP27t0rJ0+elJMnT8oT\nTzyRlufcnRwdm2Es+r179/Lee+9RtmxZAJ577jllMWfNmpXQ0FAOHjwIQMOGDbl+/bplsqZ3/P39\nAThy5AgAbdq04cSJE+r9sLAwZa0EBQV5XsBkkt7naADLR0AJERDgSJ3Stm1bevbsCUDevHkJCgpS\n/mVTgZj3Su3atZk7d65H5SxSpAhvvfUWQJLW8eeff062bNm4cOECALdv33abTBMnTgSgVKlSLvvr\n1q1L27ZtAZSnwZNoH71Go9FkcDKMRX/v3j3Gjh2rtv39/cmcOTPg8M+9/vrrlC9fHnD477VFnzg/\n/vgjp0+f5ujRowAu1rxJpkyZ1F/ztRlVYCWm1QTw3XffWShJ4phRQekNX19f2rZty5w5cwCH1e4c\nKXL79m2XyLC8efNStGhRAHr16kWOHDkA6Natm9tlzZEjB5s2bSJXrlwADBkyJF6f6tUdCXMbN24M\nwNdffw3A4cPuKYtRoUIFQkJcs7GPGzcOgDt37nh8xONMhlH0cbl69ap6/dFHH9G7d2+1nR4UUnrm\n448/TvL9l19+WQ3vncMtrVxlXbFiRQBatmypvvvvv09/6ZUee+wx3nnnHavFcMF0dy5ZsoTSpUu7\nuGf+97//AbB8+XLWr1+v3HngUO6TJ09WfT0RTJA1a1YlT7FixTh9+jQAd+/edelnGAZ169YFHMZI\nVFQUy5cvV7K6g/79+6sHD8DKlSvVZLHNZnPLOZOLdt1oNBpNBifDWvTOBAQEuDzF00N+n0eZIkWK\n4O3tbbUYLpiWnpeXlxpluHPSLaUULVqUIkWKWC2G4qOPPmLYsGGAwwqOiIjg7NmzAIwZM0ZZwXFp\n1qwZH374oXLb2e12rly54nZ5zdFQo0aNAOjbty8Af/31l0u/7NmzK7cJwPz58/ntt9/cJpe/vz+V\nKlVS26dPn2b16tXpJiAgQyv6Bg0aADBz5kzgvjsnKirKKpEyJEuWLAHg119/tUyG559/3rJzp5QT\nJ05YvpJ42LBhyvC5cuUKAwcOZN68eQn2DQgIUP77Z599FhFRD9U//viDMWPGuFXWCRMm0K9fPwBi\nYmKYOHEiP/74o0sfMwps/Pjxat+2bdvUce4iKChIzQEC7Nq1i1mzZqntqlWrurznaTKsoi9evDjL\nli0DwM/PDxFRi2dOnTplpWiPNHnz5qV169Yu+z7//HMAS5WWmaIhvWNaouCYFLx165Ylcnz77beA\nw4o3/fC9e/d2mWwtW7asGn0MGzaMunXrqoeCOc9l/qaaN2/uFou+ePHiStbg4GCyZHGorKioKHXf\ngeO+HDVqlModkzdvXuW3f/XVV+P58N1N2bJl+e6772jRogXgGGmao59//vmH69evq8WdzvMe7kL7\n6DUajSaDk2Et+tq1a7vMgIM1Q6bEKFmypNUixKNOnToA/PLLL/Hee/nll4H7IWomv/zyC1u3bnW7\nbBmFwoULK3dHWFiYZXKYETLO81V169ZVDRwL5cwEbHFWsgOwbNkyFc3mLv/8kCFDqFmzpto2/exj\nx47l3LlzrFmzBkBZx86YaU88EWVXq1Ytl/MEBwcTHBzs0se06L28vMiRI4eSb9GiRWqexIwiSmsy\nrKIfNGiQuvCZMmXi+PHjylefHjCHdFZirhyePn06/v7+6sdvDiXNv+PGjVMrIONmr4yr+K0ga9as\nFCtWTG17YiicUkJCQtQc0RdffGGxNK5hiPXq1YsXO++svJxfDx8+nNGjR7tVtqZNm/LSSy+57DNX\nZBcpUoQDBw5Qrly5RI83Hz7uUp7ObNu2Lckgj1u3bql1BnH7dejQgW+++QZwn6zadaPRaDQZnAxn\n0ZvDpcKFC6snp91uZ9SoUR55sieX48ePq5l4q8I9TeuoTZs2wP2hpbkYyrT0evbs6RJGl97w9fWl\nVq1aanvDhg0WSpMwZm7/QoUKER0dbbE0qAiZLl26qO//ypUrKoABHKuMzXvBvEdNK96d1nzOnDkB\nR96YuGG8mzdvdtk2F2yFhIRQsGBBtf/3339X4ZX//POP22RNjF27djF16lS1vX//fqpVqwYkPJIz\nV/b+/PPPbpE3xYreMIxAYA6QD0cWtWkiMtEwjDzAQqAocBroKCIeyzdgxtWabglw+PU2btzoKRGS\nxZo1a3jxxRctO/93331HkyZNAIiMjOT33393GbInN967dOnSbpUzOcSNuFm3bp1FkiROoUKFAFTU\niNWY8fEJxcmb/u527dp53F1jnhfiJwYDVOqSDRs2sGDBApUUrmzZshQsWFDNF7355ptujZt/EDdu\n3OC7775zUdr79+8HHA/U119/XYV/AyoleNmyZd0id2ruuhhgkIjsNQwjJ7DHMIww4GVgo4h8bBjG\nUGAoED8RhRsoVaoUL7zwQrz9zZo1IyIiwhMipBhPW6H58uXDz88PcFhJZj4QkxEjRvD+++8/8HMG\nDx6sLKmPP/5Y5cfxJP/973/V6x9++CFd5qB3Jr3OIQQEBDBs2DA10hMR9bvp1q1bvJh1d3H+/HkA\nwsPDWbt2LceOHQMck9dnzpwBHIoUUNk1TaU5YcIEAEuVPDjy69SsWZOff/5Z7YuJiQEc606aN2/u\nUXlS7KMXkb9EZG/s65vAH0AhIASYHdttNtAmtUJqNBqNJuWkyTjSMIyiQEVgJ5BPRMz1yBdxuHYS\nOqYX0Cstzm8yYMAANbPtTHq05s3MeiZ///23R85rhlCWKVNG+d0LFCjAli1blE8+IT+82TchzNDL\nFi1aMGTIELVS1lMrkJ0XIV2/ft3yBFIPYunSpVaLkCCVK1emf//+Lu4ac1GSp6x5uD+6rVixIteu\nXUt0DqtEiRIuo7l3332XnTt3ekTGuBw/fpzvvvvOxR07b948Nf+xa9cuFaq6du3aeOHVptzuylWf\nakVvGEYOYCnwloj87XyTiIgYhpHgtyQi04BpsZ+RJrORvr6+LjdpYpNe/v7+REZGWqoQ4k4ymbla\n3I0ZI3/kyBE10Wb6Qk0FH1fRX7lyRf3Qv/76a0JCQpSLLF+++8/xPHnyMH36dPXA6N+/v9uVfb58\n+fDy8kr3GUk7duwIOHKymGkE0gumT3zy5MmIiApLHDNmjCqkYQXOGWjjki1bNsaPH0/hwoUBx2/9\nl19+4eLFi54Sz4WoqCjGjBmj5r3y5s1Lvnz5VHnD8+fPq7TpzqkSTEJDQ9XnuIVUlgD0AtYDA532\nHQUKxL4uABz1RClBX19f2bVrl0v5rpkzZ8rMmTOlYMGCUqFCBalZs6bUrFlTrly5Io8//rglpc/M\nFh4ebmkpwf79+7uUC3QuCWhuHz9+XI4fPy4VKlSId3zRokWlaNGisnv37njHme2ll15y+/+xcuVK\nl+vYrFkzS7/XhJphGLJ69WpZvXq17N2713J5nFuzZs3U92eWB2zXrp20a9fOctmSatWqVZPbt2+r\n771NmzaWywTI0KFDZejQobJhw4Z4JU3N6xx3/7Rp0yR79uySPXv2lJwzWaUEU+yjNxwm1NfAHyLy\nidNb3wPdY193B1am9BwajUajST2pcd3UBroBBwzDMKe43wU+BhYZhtEDOAN0TJ2IyePWrVusWbNG\nFaAA1Kq6Bg0aEBQUpKIxIiIiPJ7kKC4zZ85MVlSLu/jyyy+V79NMUma6PyIiIhgyZIhKUnbp0qV4\nx5trEpo2barC8SZOnKgiedyNOWSvXLkygAqfXb9+vUfO/zD4+/urldBr1661WBoHcd014HDRjRkz\nxiWWPr1hzhUNGjQIb29vJXt6SWpnFu1Zv349GzZsSPT3EBkZqdw1n3zyidtj/VOs6EXkFyAxx2ij\nRPa7la+//lrFfjsXBy9SpAjXrl1j8ODBgCMHRmRkpBUiKuKGIXo6dXJMTIyaaHPOAviwXL161SWt\n7TfffKPCB92ZA+fxxx8HUKGds2c7Ar3SY62Bjh07qoeoJyc1E6NKlSosXrwYcChOc06mQ4cO6bJY\nuTPmg71Dhw7A/XvXXDiVXti3bx/ff/+9ivuvUKEC27dvBxxG6cSJEz26cE6nQNBoNJqMTmomY9Oq\n4YZJkQEDBsjff/8tf//9t2zatEnq1Klj+USNc+vSpYuaSAoPD5eyZctaLtOj1CpVqiSVKlUSu90u\nW7ZskVy5ckmuXLkslyuh9vbbb8upU6fk1KlTUrRoUUtlKVu2rCxdutRlgnDUqFEyatQoyZs3r+XX\n6kEtLCxMwsLCxG63y/bt2yVPnjySJ08ey+WysCVrMtZyJe8uRa+bbrrdb0WKFJEiRYrIpUuXRERU\nBEh6j66J25YsWSJLliwRu90urVq1slyedNCSpeiN9ODTTKs4eo1GkzDmYp1hw4bx7rvvqkpSCcV0\nax4p9ohIlQd10j56jUajyeBoi16j0WgeXbRFr9FoNBqt6DUajSbDoxW9RqPRZHC0otdoNJoMjlb0\nGo1Gk8HRil6j0WgyOFrRazSaDMG6devUSlAz6ZnGQfooSa/RaDSpoFatWjRu3Fhl4kwP64PSE9qi\n12QosmbNytixYxk7diwiolLDajImgYGBBAYGMnXqVAzD4MqVK1y5csVttVcfVbSi12g0mgyOdt08\nIjz77LMALF++nC+//BJAFVLR3Kd9+/a8/fbbgGP4nh4qTvn6+jJixAgAqlWrxvHjx9V70dHR6vs8\nduyYJfIlxbvvvgtA9erVadWqldq/ceNGVQjbSr7++msAypUrB0C/fv0A+O233xI95l+J1SmKdZri\npJuXl5dMmjRJbt26Jbdu3RKbzSadOnWSTp06WS5bempmsXLzGtlsNjl69GhKCy6nuj3xxBPyxBNP\nyG+//SYTJkxQtQfu3bunXpv54G/cuCE3btyQ//znP5ZfR0AmTJggEyZMkBs3bihZzWtqtsOHD0vh\nwoUtlbNhw4Zy584duXPnjthsNlmzZo08/fTT8vTTT1t+DT3Y3FscXKPRaDSPBql23RiGkRnYDZwX\nkecMwygGLAD8gT1ANxGxthL3I8Jjjz0GQNGiRXn++ecBaNOmDWXLllU1R+fPn8/KlSvdKke+fPl4\n77331PZrr71G9uzZAVi0aBGXLl3iwIEDABw+fJi//voLgJMnT7pVrqQwr1e2bNnUvk8//ZTbt29b\nIo9Zy7ZgwYIMHDiQe/fuATBlyhRVKNzX15dcuXKRM2dOAMaPH8+JEycA2LBhgwVSQ4sWLejVq5eS\nLzFKly7NzJkzlUvR0wQGBjJ+/Hi8vLwA2LNnD88//7xl37e7yJ49O6VKlVK1Z8+ePZuiz0l1mmLD\nMAYCVYBcsYp+EbBMRBYYhjEF+F1EkqzcmxZpimvWrMmaNWvYv38/AI0aNSImJsalT+bMmQGoU6cO\n27dv5+5d654/xYsXp3bt2mo7W7ZsjB49GoC8efPGCw/btm0b4Ch6fuPGDbfIVKBAAQBWr17N008/\nnezjzp8/D0BQUJBb5HoQZcqUYdeuXcD9Ahtw//u2gqxZswIOhfTKK6+oYvArV64kUybHQLpWrVr4\n+Pjwf//3fwCUKFFCFYl/9tln2bFjh0dlLlGiBLt37yZXrlxqn2lgxL0fIyIi6Ny5M5s2bfKojIGB\ngYBjrqpixYpcvnwZgE6dOrF582aPypJSSpUqRevWrdV2rVq1KFasmEsf87pny5aNkiVLKkWfN2/e\nuB+XrDTFqbLoDcMoDLQERgMDDYd0zwCdY7vMBkYCbivRbv6wQ0ND8fPzo27dugCsXbuWEiVKxJUX\ngCJFijB79mxef/11AP755x93iZcow4cPp3v37kD8H5EpJzgmm4YPH66sZndSvXp1AKXkFy5cCMCW\nLVvUhCFAVFQUb775JgDBwcHcvHnT7bIlRY8ePVysz61bt1oojQPTiDhx4gTvv/9+gn3WrFlDq1at\nyJEjh9pnvs6fP7/7hYwlJCQEgFGjRrkoeXBMugIUKlSI0qVLq/379+/3uJIH+PbbbwGoWLEiAG+8\n8QbAI6HkJ092qMGXX35ZGQLg+L0npgPM/Xv27EnVuVPro/8MGAzYY7f9gUgRMU3pcKBQQgcahtHL\nMIzdhmHsTqUMGo1Go0mCFFv0hmE8B1wWkT2GYTR42ONFZBowLfazUuy6+eKLLwCoUaOGy/5GjRol\neVz37t2ZNm0agCWLaqZPn865c+fUdrFixdi3b5/aXrJkCeBwi8R1QbmLFStWADBp0iTefPNNwsPD\nAbDZbC79xo8fz5w5czwi04No3LgxAwYMUJbPrVu3ePXVVy2WKnGefPJJ2rVrp7ZHjBihXDmAcj3+\n8MMPHpOpb9++wP36sWfOnAFgzpw5jBw5EoDChQvz/fffU6FCBQAqVapE69at+f777z0mZ//+/alW\nrZranjNnDmFhYR47/8OSM2dONR/z3Xffqf3Hjh3jyy+/JDIyMlmfs3r1auW6STGpCIn8PxwW+2ng\nIhANzAOuAFli+9QE1rsrvLJ3794q3MsMA0tumzlzphiGIbEPGY82f39/2b59u2zevFk2b95sWQhg\nYi1z5szyww8/xAups9lscvr0aSlVqpTlMpqtefPmLvKdPHnScpkSaq1atZJWrVrJ+fPn492Lc+bM\nkTlz5sibb74pwcHBEhwc7HZ5cubMKTlz5pTPP/9cLl26JJcuXZLbt2/LoUOHpHPnztK5c+d4xwQF\nBcnZs2fl7NmzYrPZ5LffflOf405Zn3vuOXnuueckOjpafc+bN2+WHDlyWP69JtaqV68umzdvdrk3\n9+3bJ/v27ZNChQql5bncG14pIsNEpLCIFAU6AZtEpAvwE9A+tlt3wL0hIhqNRqNJEnesjB0CLDAM\n4yNgH/B1Wp+gcOHCgGPyyHniEu6viFu3bp2aEJs/fz47d+7Ez89P9Tt27JhliY9eeeUVqlWrxrBh\nwwDSXUiYzWZjwIABBAcHA46JOJNXXnklXa3gbNq0KYAaBrdv3z6p7pZhRosUKFBAJd46duwY0dHR\nyj3iyfDUxo0bA9CnTx+178iRIzz55JOJHnP27Fk1CT9ixAgqVKigXKSm2y+tKVCgAP/9738B8Pb2\n5sqVKwB8+OGHKkIpIXx8fNSEbePGjVm3bh07d+50i4wmmTNnViuJ33rrLXLnzs3Vq1cB6Natm6Wr\ntNNE0YvIz8DPsa9PAtWS6p9azDA+f39/l/0TJkxgwoQJACrsyiSun/nUqVNulDBpfvrpJ65du0bH\njh0BmDhxoiWRP0lx7Ngx5at1VvS3bt2ySqQEMRV7REQEAHv37rVSnEQpVaqUej1//nwAXnrpJavE\nwdvbW702fxvLli174HHmGo5bt26xfv16l89xB0uXLqVKFUf04NWrV+nc2RHQZ0YDOePr60vDhg0B\neOedd6hTp456b9iwYbzzzjsALF68mIsXL6a5rJ06dVIPbcMwCAsLUw9Gy40jq9MfpMRH/9RTT8lT\nTz0lt2/fliNHjsiRI0fkxRdflEyZMiXYv2XLli4+0evXr4u/v7+lPrzBgwcreebNm5fu/PSlSpWS\nyMhIiYyMdPEzVqtWzXLZAOnatat07dpVRETsdrvMmjVLZs2aZblcibWOHTtKx44dxW63y+3bt+X2\n7dvy3nvvSdasWS2R5/Dhw3L48GGx2WwycuRIGTly5EMd37hxY7HZbHLo0CE5dOiQW2Rs3bq1Sm9g\ns9mkWbNmifYtW7as/PzzzwnOK505c0YiIiLU9vjx490i79tvv61+0/v37/fUb1qnQNBoNBoNWG7N\npybqpmTJkpIjR44Hzr6/++67Lhb9uHHjPPGkfWDr3bu39O7dW27evCnz5s0Tb29v8fb2tlwuQP7z\nn/8oC8g52mHAgAGWRCrFbf3795f+/fuL3W6XGzduSLly5aRcuXJSvXp1yZ07t+TOndtyGZ2bGeG1\nYsUKl0RhixcvlgYNGkiDBg3Ey8vLI7IEBwfL5cuX5fLly2K326VRo0bSqFGjh/qMxo0bi91uVxE7\naRkp5O/vL/7+/rJjxw6x2WwSFhYmYWFh4uvrm+gxbdq0iRfh0r59e2nfvr0ULVpU6tevr967ffu2\nW65rjx491Dnu3Lkj06dP98T3mSyL3nIlnxpF/6Dm5eUlXl5esn37dhdF37FjR098AcluW7dulaio\nKClfvryUL1/ecnkA+frrr9VNGxoaqhSDzWZLF0r0hx9+kB9++EHsdrts27ZNBg0aJIMGDRKbzSbn\nzp2Tc+fOybFjx2TevHnqvWzZslkud758+WTVqlWyatUq+f333yUqKkrdlzdu3JCGDRtKw4YN3SrD\nV199pb7bjz76SLJkySJZsmR5qM8wXTdm69atW5rJN2bMGBkzZoyLyyYptw0gGzdudJGnffv2Lu+X\nLFnS7Yrey8tLNmzYIBs2bBC73S5RUVEyffp0dyt8rej9/PzEz88vXtxyelP0xYsXlxs3bsjs2bNl\n9uzZlssDroq+fPnyKg7cZrNJz549LZfPWdGPHDlSatSoITVq1BCbzabmFiIiIuTevXvq/9i9e3e6\nWgNQrVo1CQ8Pl+vXr8v169fFZrPJnj17ZM+ePUlar6lt4eHh6ppUrVo1RZ/xzjvvuEXRt2rVSu7d\nu6e+t+XLl4uPj4/4+PgkedyKFStc5NmwYYM0b95cmjdvLn379lXzETabTS5duuS2a1ugQAEpUKCA\nDB06VI2Go6Oj4z140rBpH71Go9FowHJr3pMW/d27d+Xu3btSsWJFtz3RU9ref/99VVzEalmyZs0q\ne/fuVRZQQECAFCpUSAoVKiRnz56VVatWWS6js0Xft29f8fX1FV9fX2nWrJkEBARIQECAAPLCCy+4\nRA+dO3dOihUrJsWKFbP8f4jbFixYoO7VFStWuO084eHhsn79elm/fv1DR4aY190s8HL8+HE5fvy4\n+Pn5pYls//3vf10s88DAwGQdV7t27QQjbhJq77zzjke+z0mTJrl4Etx0nmRZ9P+qUoJmTnDnnDLu\nYtKkSWzfvp2lS5cCD86QeefOHZVLPXv27JYuovLy8lI5TUzMVMQXLlygVq1aBAQEAPfj1z2Nmc3T\nMAxq1aqlch6tW7fOpd/ChQtZu3Yt4MgAWKJECTp06ADAuHHjUnz+NWvWqLjzRYsW8ffff6f4s0yc\n13bETVub1pj1BcqWLZustQcNGjSgatWqNGjQALif9/+zzz4DSLPU2XF/J6tWreKnn3564HFm/v+E\nEBEuX77Mxx9/DKRtdlNzTc/t27fj/RZGjRqlrle5cuVUnn8zx5Yn0a4bjUajyeBkaIveXEVnkuoM\ncA/B4cOH+fbbb1UaATPdQXLInTt3ukuL4Iy3t7da0m+VRX/69GnAYa3Vr18/0RFGjhw5VMZDX19f\nRCTJyknJpXDhwsoyGzJkiFrleuPGDUqXLs3q1auB+6PIpDCtUefi2+aqZHdhFr358ccf1epNs7qV\nSdOmTalfvz4A9erVS7CQiznSSyvGjRunMtGGhIQQHBysfkPJwcz0arfb1e/9ww8/VLng0xrTOzBm\nzBhCQ0Nd3ouIiFA1HT744AOVtdQKiz5DK/qnnnrKZdtd+TgSYsGCBQwdOpQhQ4YAjuXuPXv2BBzD\nfmdy5sxJ7969lTvCE0VGkkJEiI6OdqnW5Ez27Nl54oknAOtSDpjpfMFRpOPTTz8FHD+ookWLAg5l\nVqtWLZeU1RcuXFDutNQQGhq9TDgZAAAgAElEQVSqHiA9e/akd+/egKOkYfbs2ZWizpEjBzNnzuTw\n4cMA/Pnnn9y5cwdwPBReeeUVmjRpAjhSGJuKY9SoUamWMTk89thjzJ07N9H3zRTKZn4eZ7Zt2+aW\n/DGmQhw8eDBeXl5UqlQJcJTVTIyzZ88yY8YMFi9eDHgu5YBZ/jMuderUoV27di4ps61Mu2L5RKy7\nJmOLFSumwtbMyZAOHTpIhw4dPDIRA0ihQoXkjz/+kD/++EPFSd+4cUOmT58uzZo1kwoVKkiFChVk\n8uTJcufOHbVwxlPyJdWcwyu7d++u9puLWMwl/VbLOW7cOLl27VqCk27moiRzEv6XX36RsmXLprkM\nn3zyiQwfPlyGDx8u0dHRYrfbZdOmTbJp06aHTp/9wQcfyAcffODWa/bJJ58ke+LSJO7+AwcOSIEC\nBSz//q1uJna7Xf78889436eZ7mLy5MnukkGHV2o0Go0mA7tucufO7ZKWGO77dT3F+fPnadmyJeCo\naDVgwAAAXn31VV599VWXupCXLl1SBaTTG8WLF3fZvnbtmqUpV50ZPHgwU6dOVVlLnYsuz5s3j4iI\nCOWy27Jli1tkGDhwoHo9fvx4WrRoQadOnR76c2bPns306dPTUrQEmTZtGm3btgWSX9D97t27/PPP\nP8pFtm/fPstdjOkB0y/ftWtXihUrZnoo2L9/P9evX2fs2LEAlv9eMqyiTy+YOcZHjBjBkSNHABgw\nYACVK1dWfRYvXswnn3ySrn44zuGCffv2Valsq1atyrJly9IsnC4tOHHihFJcVnP79m2WLl2q5mGO\nHDnCiRMnVGHt2rVrq/sAoEyZMnz++eeAIwVwciZvU8uRI0dUHv927drx7LPPAo6J2fbt26sU32Yf\ngA4dOqgJZs193n77bcCh8M2AAHCdQ0oPaNeNRqPRZHSsnoh112RsixYt4k2MVK1aNcW5Pf5tzc/P\nT9asWSNr1qyJNxH3/vvvWy6fbrrphvBvXxlrDkdN7t27l+6qOKVnbty4wfvvvw9A/fr11UrIxYsX\n8/XXaV4dUqPRuJNUWuK5gSXAEeAPoCaQBwgDjsf+fcwKi75Zs2bKkj9z5oxLiKBuuummWwZpHgmv\nnAisE5EyQAUcyn4osFFESgIbY7c1Go1GYxGGGQ700Acahh/wG1BcnD7EMIyjQAMR+cswjALAzyJS\n+gGflTIhNBqN5t/NHhGp8qBOqbHoiwERwCzDMPYZhjHDMAxfIJ+ImHGCF4F8qTiHRqPRaFJJahR9\nFqASMFlEKgK3iOOmibX0E7TWDcPoZRjGbsMwdqdCBo1Go9E8gNQo+nAgXETMrEZLcCj+S7EuG2L/\nXk7oYBGZJiJVkjPs0Gg0Gk3KSbGiF5GLwDnDMEz/eyPgMPA90D12X3dgZaok1Gg0Gk2qSG0c/ZvA\nPMMwsgIngVdwPDwWGYbRAzgDdEzlOTQajUaTClIcdZOmQuioG41Go0kJbo+60Wg0GkupWbMmNWvW\nZNu2bYgIZ8+e5ezZs1aLle7IsCkQNGmLmYr3rbfeIjw8nFq1alkskSv58jmieLt3765K8pnl8pzT\nQU+YMIHBgwdbI6QmzQgMDGThwoXUrFlT7du+fbuFEqVzrE5oltoUCGPHjpWxY8cmWK1n/vz5Mn/+\nfKlfv77Vy5RT1Xr27CmTJk2SSZMmyb59++TMmTNy5swZefzxxz1y/kWLFonJ2bNnJTQ01PJr4txC\nQkJk586dsnPnzgdWTLp796506tRJOnXqZLncPj4+smrVKnVtd+3aJd7e3uLt7W25bOm1mZXN4hIa\nGiqBgYGqWS2nv7+/jBgxQumimTNnuutcusKURqPRaB7RydgyZcoA0KVLF9555x0AvLy8Eu1//fp1\ntmzZwn/+8x/AUZ09PZMnTx5VSPyZZ56hQoUKPP7444CjsMWsWbMAGDJkCLdu3XKbHKa7JjQ0VA2L\n05vLBhwFyitUqJDge/369SMmJgaAFi1aUK1aNa5cuQJAcHCwx2R0JmfOnADMmjWLNm3aKNfSzZs3\nlbvp0KFDlsiWngkMDHTxv7/wwgvqvjx37pxVYrlQr149AMaMGePiVpo4caJLJbI0JFmTsY+cjz5L\nliyMGjUKgOeffz5Zx+TJk4eQkBBy5MgBxE9hnB7IlCmTqlbz1ltvkT9/fvXeqVOn6NGjBwBhYWGE\nh4e7XZ6aNWuqMmnnzp3jhRdecPs504K7d+8yevRoALy9vZkyZYqqjjV16lRWrVpF1apVLZPPx8eH\nfv36AdCmTZt475slCP/73/96VC6A5557jpEjR1KxYkW1b+/evQCEhIRw4cIFj8vkjHk/guOe3L59\ne7pR8ABvvvkmH330EYDSNSabN29Wab8jIyNd5P7tt984c+aMW2XTrhuNRqPJ4DxyFv2IESNcLPk/\n//wTgCVLlgDw5JNPAqjIC2eqVEmf2RaqVKnC2LFjadiwodq3bds2AKZMmcLcuXMZOXIkEN9ScBft\n27dXr99+++10ZTklxKlTpwCHVbV27VqLpUmcJk2a8MEHH6jtGjVqqPrBX375JS1atABg3Lhx3Lx5\n0yMymVZo//798fHxUec9ePAgBQoUAGDPnj2UL1+eq1evekQmZ0yXR4cOHdS+F154IV3ck97e3oCj\nrvKHH36otuOyfPlyEnOT37p1i59//hlwLW6fljxyij7uhTC/fLMYr6+vL+Bw17z44osAfPzxx8B9\n3+iQIUNUdXYreemllwAYO3Ys+fLl4+7du4DDr7xlyxYAwsPDmTt3rhrmDx48mF69egEwd+5ct8gV\nGBjIwIEDWbx4MQCLFi1yy3nSijFjxqgfUXpW8uDw4Zo++dq1a7N7927lprtz5466f/38/Dyi6Nu1\na0ffvn0ByJ49O1u3bqV3796AY57gueeeAxyFy6dMmeKibD3FW2+9pV4PGjQISD+hlBMnTgSgZ8+e\nGIbhoswjIyNZtmxZoseWKlUKgLp16yp38v/93/8xbNiwtBfU6tDKhw2v/Pnnn1Wo3PXr16VgwYJS\nsGDBJI+5evWqS4jd6NGjLQ+/6tSpk1y8eFEuXrwodrtd9u7dK61bt5bWrVsLIOXKlZNy5crJhx9+\nKNHR0SpM686dOzJr1iyZNWuW5M6d2y2ymeFrixYtkkWLFll+rdKyrVq1Sl13T563Z8+e0rNnTzly\n5Ihs3LhRNm7cKP7+/i59Fi5cKDExMRITEyMNGzZ0qzzm7+bmzZvqnL/++qvkzJkzwf5Tp06V6Oho\nqVSpklSqVMlj1y00NFSFUKa3e7FXr14u4dw3btyQr776Sr766isJCgpK9ufUrl3b5XPKlSv3MHLo\n8EqNRqPRPIKum27durFw4UIAhg0blqxIgPfff58vvvhCbTdo0IDcuXMTGRnpNjmTIiAggC5duqiQ\nyVOnTtG0aVMV9lmqVCkVgdOtWzcyZ87M8OHDAVixYgUHDx50q3zVq1cH7s97ZBTq1q1L8+bNVXil\npwgMDFSRQD4+Prz88ssASfq7jx8/7laZzDkfHx8fbt++DTgiaxJzF0VFReHt7U3WrFndKldcnF1F\nptvGarJlywY45tbM3+yxY8f44IMP2LBhw0N/XtWqVV1cPsuXL+eZZ54B4Pz582kgMVjutnlY1w0g\n2bJlk2zZsiW7f+PGjeXatWsu7hsrVs9lyZJFsmTJItOmTZMbN27IzJkzZebMmVKhQgUBpF27dtKu\nXTtZunSpGsbdunVLXnrpJfHx8REfHx+PyGlSs2ZNqVmzpmVD47Ru9evXF5vNJv369ZN+/fp57Lyv\nvvqqREZGSmRkpISFhUmpUqWkVKlS8fo1adJEuVFMF547WsGCBdXvICYmRrp16ybdunVL8pjQ0FCJ\niYmRsLAwCQsLk5deesnt161mzZoiIrJt2zbZtm2b5fePu9qpU6dcdFNkZKSUKVNGypQpk5zjtetG\no9FoNI+g6wYc0QkPw4YNGzh37hx+fn5ukih5mKFXr732GnB/Nd/vv/9O1apVmTdvnupnRuB89NFH\nzJkzxyPyBQYGqtfbt293iWzo2NFRVqB9+/acO3dOuXXCw8MtCXMLCgoC4PLly1SsWNEl7PTo0aMA\nahWlmfBs+PDhnDx50m3RSgnh7e3NG2+8oSK+Ll68yLFjxxLse/z4cS5duuR2meIufkvKTWRGhpgL\nucwQ4LJly7r9vhwwYAAAn332WbL6BwYGuizyM49LD2GYSbFx40ZeeeUVwLE6ukKFCmm+gOqRVPQZ\nhSxZHJe/devWfP311+pB8L///Y+hQx3ldz0ZRua8ZLtmzZoqfrl9+/Yu78H92Obt27crxeHuH1Tx\n4sUBRxoG8/pcvXqVp59+2kXRHz58GHCEr+7fv586deoAjrmZjRs3cu3aNbfK6UzLli15+umnlQ/2\nww8/TLRvpUqV1EPJXJHqbvbu3ZvkuZYvXw5A/vz5XfzIq1atcrtsycE0Tt566614KQbM7RdeeCFd\nhgiboastW7ZU+zZt2uSWVbL/CkVfvnx58ubNq+KXrcLMuXLixAlKlCih0hqYcbM7duwAHLHNVixM\niUvcJeeAmiQ2J2wHDhyo4pzdNVmWNWtW3n77bbp06QKAzWajbNmy6v3Lly+rXEfe3t6UK1cOgN27\nd7NkyRK16OfIkSNqItRTmA9BU5kmZs079wUoWbKk21JdGIZBpkwOr23BggXV9UlIwZhrPapWrcqX\nX36p9nti5PEgAgMD2bp1q3p97tw5tfYjMDBQTeQuXLhQpUxILxQtWpTu3bsDjofo33//DbiuGUhL\ntI9eo9FoMjj/Cou+bt26LkPP8PBw/vnnH4/LYZ6zWbNmbNmyRVlS5nshISFA0mF37iQxi+fcuXPK\nJ25iDoUHDhzo4tt3B1WrVnVxedhsNjZu3Ag4XAgLFixQ4YsHDhygXbt2gGMVqnMqh6FDh3osMZc5\nLDdTGsyYMSPRvvXr1wccVryJ872R1ly4cAG73Q44rMlKlSoB8S364OBgatSoAcDnn3+OiKhRZ3pY\nWR4aGqruPdOF6Ow+NOeVFi5cSPv27dONRe/t7c3kyZPValgR4X//+x+A+6pjpTIscgBwCDgIfAdk\nA4oBO4E/gYVA1rQOr0ysZcuWTfz8/OK1X3/91SV8acKECeLr6+vxMCqzKMJnn30Wr0jKqVOnxNfX\n1xK54raESCzUctGiRXL27Fk5e/ZsmsthhiGePHlSbDabREdHS3R09AMLyXz22Wfy2WefxSs68tNP\nP3nsGnbt2lW6du2qzu286jluCwkJkZCQELHZbLJ3717Zu3dvoitU06qNHDlSRo4cKTExMXL58mW5\nfPmybN26VbZt2yZbt26VrVu3SmRkpAr3NEMxFyxYIAsWLPDINXxQeOXZs2dd7s/EPscsnOMJmUuW\nLCklS5aU/Pnzx3vPLCqzcOFCl/ty7dq1kiNHDsmRI0dKzpms8MoUW/SGYRQC+gHlROS2YRiLgE5A\nC+BTEVlgGMYUoAcwOaXnSS4dO3akd+/eKh90HFldJpIGDBhAixYtXKIGPv30UwC3WvrmBNZTTz0F\nONKTgsNibtWqFWPGjAEcyaWs5Ny5cwQGBiqf8YMmstxl0S9YsACAIkWK8OOPP6o5g82bNyd6TI4c\nORLNM1+7dm2aNm3K+vXr017YOJiWvIhw9epVZQknhJkW+ObNm6xevVq9didmIjNApc+tXr26y2/l\n2LFjql/9+vXp0aNHkiOTtGb79u0sXrxY+dpDQ0Nd5oF27Nih7r3kzGeYFn5aTczmzJlTzQe1bduW\nHj16qMVUNpuNiIgIli5dCjh0kDm6LFq0KIBaXNW2bduHjiR8WFLro88CZDcMIwvgA/wFPAOYSypn\nA/GTbms0Go3Gc6TSddMfiAIigHlAXuBPp/cDgYOJHNsL2B3bUjxU2r59u2zfvl1u3bqVaJ1Qu93+\nwFqipvuhSpUqaTaMMwxDGjZsKA0bNnSR79q1a1K3bl3x8vISLy8vKVq0qFy8eFGioqIkKipKnnrq\nKY8MMxNr5lA3OStjnevJpuVq45EjR8qdO3fkzp07EhERIdWrV0+yf+7cuSV37tzSq1cvl+912bJl\n8vfff8vff/8tNptNtmzZ4pFrOG/ePJk3b57ExMTInj17JGfOnAm6Y0qVKqVkPXHihBQoUEAKFChg\nyfdeuXJlqVy5coLvhYaGis1m86jrBhzuTvO36ezGCQwMjFc7NjQ0NMF7dtu2bcm+n5PTqlatKlWr\nVpVDhw6p727lypUyffp0uXbtWrxV+AnpoH79+kmuXLkkV65cqZUnWa6b1Cj5x4BNQADgBawAupJM\nRZ8WPvqOHTvKrVu3XJSomSVwy5YtD6XozTZv3rw0u0nr1avn4oc/ffq0nD59Wpo3b+7SL0+ePLJ4\n8WJ1w3700Uce+yEl1AYOHCgiSWevNOcbzB9hWvvoL1++LBERERIRESHPPffcA/vPmDFDZsyY4fJd\nnjp1Snx8fJS/PDIyUqKioqRly5bSsmVLt17D0NBQlTYgMcVYqlQpWb58udy4cUNu3LghtWvXtvR7\nf9D/42kfvdlMBe3sk3e+P815IlPZm835AZFW92fJkiXl+vXrcv36dYmIiJDOnTtL586d4/Vr2rSp\nS9ZZEXHRBV988YVHFX1qXDeNgVMiEiEi94BlQG0gd6wrB6AwkEZZeTQajUaTElITXnkWqGEYhg9w\nG2iEww3zE9AeWAB0B1amVsjEKF68uJr8MDHrce7cuVMVWjYxJzzMmrNmSoQhQ4aoPmmZNXD27Nnq\n9cmTJ1U41cmTJ136Xbt2jaCgIDUJZoa+WYUZhuY8Cea8DL1Dhw4ui6k++eQTt8hx4MABwLFaMC7m\ngqlKlSrRqVMnqlWrBjgWpf3xxx+AYwFSdHS0SnnQoUMHnnvuOQICAtwirzNm1SjDMDhy5IhaBQ33\nJz/NurCvv/46gFr8kx5xDv30NOb9GBQUpCZUJ0yYkGARlIQKcKdlzeNGjRqp77JFixZKtty5c9Og\nQQMV2hsSEoK3t7f6Ta9YsYLy5csDjuCF3r17q2v6/vvvs2vXrjSRL1FS6aP/ADiCI7zyW8AbKA78\niiO8cjHg7S7XTevWrWXTpk2yadOmJN0xIiIXL16UZ555Rp555hl1fKZMmSRTpkxqCJUrVy7JkiVL\nqoZSpt/9p59+ErvdLkeOHJEjR45I0aJFxTAMMQxDvL29pVixYipD3YEDB8Rutyv/XqtWrTw+NI/b\n4vo/EyM0NNQt5798+bL6/n744QeV6dNs9+7dk3v37qk+Bw8elIMHD0qHDh0S/cz8+fPLrl27ZOnS\npbJ06VK3Xr+3335b3n77bYmJiZHDhw+re2/VqlUqZDEmJkZGjBhh+XednDZ16lTLXDeJtdDQUBfX\nzdmzZ9W26VpM6yy1M2bMUK6bkJAQGTVqlIwaNUoiIiLiuYj3798vr732mrz22msun9G+fXu5ffu2\n6nflyhV1XUNCQuSJJ554GJmS5boxzCeOlRiGkWIh8uTJA8CkSZNo1KhRgtbawYMHGTx4sEfC6syc\nK5s3b6ZixYoqd7zzYhQ/Pz+Vf8Vkx44dqkSgu/PNJxczdM3Zeu/QoQOffPKJSmrmrkUoffr0IXv2\n7C77zNGbc81VcCS9+vzzz4GEl/E7U79+fRX66DySS2vMxW9meJ2Jc/jif/7zHzZs2OCW3CZpTWho\nKG+99RY//PAD4EjTYab0+Dfx+uuvu6SCMJMPnjx5kq1bt6pR6Pz584mOjla5/uPStGlTla+pfv36\nLuHfv//+u1rElgz2iMgDi2HrFAgajUaT0UmN6yatGmk0rKpRo4ZL9MXcuXNl7ty5lgwr+/TpI1u2\nbIm3AtZsR48elfHjx8v48eOlVKlSD1VIRbf038wiM99//72Lq2bTpk0yaNAgGTRokGTNmtVyOZPb\nzKgbs1WtWtVymaxouXLlklq1aqlWsWJFqVixYqo+s23btrJjxw7ZsWOH2Gw2GTdu3MMc797wyvSo\n6HXTTTf3tLiKfvz48ZbLpBuCrjCl0Wg0GuDRn4zVaDTup3DhwoSFhbFnzx7AkQnUXfnyNQ9FsiZj\ntaLXaDSaRxcddaPRaDQareg1Go0mw6MVvUaj0WRwtKLXaDSaDI5W9BqNRpPB0Ypeo9FoMjha0Ws0\nGk0GRyt6jUajyeBoRa/RaDQZHK3oNRqNJoOjFb1F/Pbbb+zcuZOdO3fi6+trtThJMmzYMPz8/FTp\nRY1G82ihFb1FiAiVK1emcuXKfP/991aLkyB58uQhT5489OnTh1q1alGrVi2rRcpQdO3aFZvNxqFD\nhzh06BBPPPGE1SIlSv/+/RERunTpQpcuXawWJ1lkzpyZzJkzU7x4cZdWuHBhq0XzOFrRazQaTUYn\nGUVBZgKXgYNO+/IAYcDx2L+Pxe43gM9xFAbfD1RyZ+ERu92uCiHELcCbUOvVq5f06tVLHZPS86ZF\n27dvn5Lj9u3b0qhRI6sLGMRrL7/8srz88suyceNGVUg9bp+SJUuqAslWyRkcHCzr1q2TdevWyYoV\nK5J9XKtWrSQyMlIiIyOlT58+Hpd706ZNqsj5vXv35ODBg5Z/54m1FStWiM1mk0uXLsmlS5ckODjY\ncpnitkaNGql7duXKleqecC7YbbPZJDIyUj799FP59NNPJXv27G6TJ3PmzC6vvb29xdvbW/z9/aVL\nly7Sp08f6dOnj0RFRUm/fv2kX79+KTlPsgqPZOHBfAN8Acxx2jcU2CgiHxuGMTR2ewjQHCgZ26oD\nk2P/ugW73Y7dbk/RcemJrFmzUqBAAavFiEfPnj0BWLZsWbxrZhgGAFOnTuXxxx8HYPjw4Z4VEChb\ntixr166lYMGCALz22msP7P/mm28C0K1bN0aPHg2g/oe0YuTIkQCICD///DObN29+4DGlS5dOUxnc\nQd68eQGoUqWKKoRtFfXq1VMFtgsXLkz58uVdimwnRs6cOenXrx8A+fPn58UXX0wzmSpUqADA6NGj\nyZ8/P5cuXQIgX758FClSBAB/f/94x7Vu3RpAFblPax7ouhGRLcC1OLtDgNmxr2cDbZz2zxEHO4Dc\nhmGkPw2m0Wg0/yKSY9EnRD4R+Sv29UUgX+zrQsA5p37hsfv+Ig6GYfQCeqXk5O+9915KDnNhzJgx\nqf6MlFK0aFECAgIsO39yKFKkCFWqOOoZDB48ON77Tz31FAANGjTgnXfe8ahsAJkyOWyUkSNHUrBg\nQT788EMAZs+enegxwcHBfPbZZzRs2BCA/fv3M3XqVAACAwPTTLbx48czcOBAwDF6fPPNN5Wld/78\n+QceC1hyTR8lChYsyGeffaauq8mNGzcA+Oeff9S+devWcezYMZd+7dq1Axz3b1piWustWrRI089N\nLSlV9AoRkZRUiBKRacA0ePgKUyEhIer18uXLAVixYsVDnX/lypUP1T8tyZo1K15eXsr9kSlTJurV\nq8fcuXMtkykur7zyClevXgXg4MGD8d433R8Ac+bMife+OwkODmbChAkANG7cmIkTJzJ58mQAbDZb\nosdVq1ZNKXmAo0ePEhkZCcD169fTTL64Q3M/Pz+yZEneTy2hYb2VzJs3D4Bnn30WgHPnHHbcwoUL\nLZHHdB0dOHCA3Llzq/3ffPMNYWFh/O9//wMe/ED9v//7P7fId/bsWcDxoPH29nbLOVJCShX9JcMw\nCojIX7Gumcux+88DzqZR4dh9aYqpIDNnzqyU0ZUrV5J9fHh4OLdv305rsZLNzZs3iY6OVv5Eu92u\nrOf0Qv369Vm9ejVw30pyxvzhX7hwgVu3bnlEJtOKnzBhAk2aNAEcD/gBAwYkeZypHLp37+6yv0qV\nKuozk3pAPCyGYajPfdh+5r2dXsiTJw+AUlrmiCM6OtoSefr27QuglLyp2N944w0XK94q/vzzT8Ah\nz4ABA9i1axfgePDs3r0bcMj+119/sWTJEsAxZ3DkyBG3ypXS8MrvAfNX0x1Y6bT/JcNBDeCGk4tH\no9FoNBbwQIveMIzvgAZAXsMwwoERwMfAIsMwegBngI6x3dcALXCEV0YDr7hBZpYtWwZAxYoVKVOm\nDABlypRJ9lNxzJgxbn+CJsVff/1FZGSki1+4ePHiVKpUCYC9e/daJRrgsNbr1avHu+++m+D7BQsW\nJEeOHIBjaO8p665OnToANGnSRPlcX3755SSPMQyD0NBQl+NPnDgBQNOmTdPUkgfIli0bvr6+KkrJ\nbrcze/ZsFX3hjIjEi2b6+OOP01Se1FC5cmV1zcAxEv39998tk6dGjRoMGzZMbdvtdtavXw+QLqx5\nZ2bNmsWsWbMSfb99+/bkzJlTbW/fvt29AiUnBtPdjRTGqcbExMjdu3fl7t27smnTJpk8ebJMmTJF\npkyZIl26dJGuXbtK165dJSgoSIoUKSI//fST/PTTT7Jp0yYJCgqyNOZ3wIABKo7eZrNJTEyMNGnS\nRJo0aWJ5PPLo0aNl165dicbOjxkzRux2u9jtdqlbt65HZPLz85Pz58/L+fPnxW63yxtvvCFvvPHG\nA4+bMWOGktVut8vRo0elSJEiUqRIEbfI2aBBA7l3756K2b53754MHz48wb5x4+jv3btn+XcPSEBA\ngAQEBMiPP/7oEn++bNkyS+Rp166dtGvXTo4fP+4iT2hoqOXXKiXt6aeflqtXr6p7cteuXZIlSxbJ\nkiVLSj4vzeLoHwnq1KlDnTp1lM/TecL2ypUrZMqUiZIlS6p9/v7+auLECh5mTsFTZMuWDYAOHTqw\nevXqRNcbVK1aVcn/yy+/eES2evXqqbUGP/74o5okTIqiRYvSqVMntX3q1CkGDx7MmTNn3CZnXO7c\nucPp06eT3f/bb78FHJPd5kSxpzEjUho1aqT2rV27lh49elgij7nWoXjx4i77mzVrxrZt2wBYunSp\nx+VKCDOaL1euXISEhKhJeOf4fl9fXxdrPigoSI2Q3fWd6xQIGo1Gk8F5pC36hCIbzH3OceoBAQFk\nypQp3a2IdQ6vtNvt1PtzjkYAAAuoSURBVKtXD4CwsDBL5GncuDEATzzxBFu2bEmwT3BwMLVq1VLx\n6slZiZgWdO3aVb0OCAhIcl7AtPzWrVuHj4+P6vvxxx97PKz20qVLDxV+2rlzZwCmT5+e6HfgTvz9\n/enTp4/avnnzJuCwVNMyBPVhSCwJWZkyZdR92LlzZ55//nlPipUgFy9eBKBLly4UKlRI6ZyDBw+q\ntScAUVFR6vvduHGj20dvj7SinzJlipqMdZ40goTTHKQ3Re8cXikiloZYent7q2X7t27dIk+ePCps\nMSAgQKUICAgIIHv27Dz22GOAY2l3QhONac2kSZOUO65ixYpqqfg333wDoNxyTZs2pXbt2oDDdfPL\nL7/QrVs3AI+4bEz3oWlwFCtWjKVLl3L06FEAhgwZot5L6H40HwpWKHmAr776iieffFJtmykGTBeJ\nFeTL51iPaabkMGnevLkK823bti3ff/+9SmfgqZDfuJj35eTJk8mcObNyg3Xo0MFF0Q8bNowvv/zS\nY3Jp141Go9FkcAxPDb2TFCIFK2tNErPo4xIUFKSsE4Dq1auzb9++lJ4WcFiMDzPR5kyBAgVUUqjH\nHnsMEVEum+bNm6dKrpTg5+fnMjS32+0uIWuXLzvWxAUFBbks6rl586aylDt37pzgKtq04rPPPgNQ\nCakexKlTpwgODvbo4p4TJ04QFBSUpNWe1HuffvopkHDaCXdTs2ZN1qxZQ65cuQDHymFzAtbt4X8p\nIFeuXGpSvkWLFhiGoUZ9q1atslI0hWm1v/HGG2oE379/fyZNmpRWp9gjIg90BTzSrhtAxcM/KC7e\nx8dHvXZW+KkhNcPDv/76i7t376aJHGnBrVu3GDJkCOC4VsuXL1fuhixZsqj/dfv27VSoUEEpW7vd\nripPlS5d2q2K3nQlHTx4UEViGIahVr6amMvfGzdu7PEVnJ9//jndunWjYsWKKTreCgVv8tRTT5Er\nVy6ioqIAx0pic2VneuTvv/9m+vTpQPrLLQOONR5vvPEG4HDTmu64NFTyyeaRV/TJJTo6mvDwcMAx\nKWcqsdQQERGRquPNJdEtW7ZMtSypJSYmRi1vj8s///xD5syZAYe/9MSJE8yYMcOT4gH35zRmzJih\nzp8lSxYmTJigHjw2m00tOjp16pTHZZw4cSJz586lWbNmgCNnENxX4FevXlWThmPHjvW4fAlRs2ZN\n4P5iLVO5p2clnxDpKX1EmTJlmDp1qrpnt2/fTq9eKcrhmCZoH71Go9FkcP41Fr0zQ4cOZfny5an2\n0acWcyjXqlUr7HZ7urJI4mKGLBYtWpStW7daLM19vLy8XEIvp0+f7tFohoS4evWq8h0ntLDr2rW4\n5R2sJTg4GHD4vC9cuJCsxWjupEaNGur1/v37k+1+Sw/zjdmzZwdgzZo1eHl5qe/ajASzin+lok9v\nmOGV6eFGTQ47duywWgTF3LlzyZMnjwr/S6v5F0+QnAyX7qZmzZouLqQ2bdqwZ88eCyW6H8opIhw4\ncICNGzcCsH79en788UcrRUsSHx8f5s+fDzgMInBdoW8l1t9pGo1Go3Er2qK3kJSGZlqBOfQ0DMOy\nFZLOmMP75s2bExUVxeuvvw44IjEeFdLDAr633npLhVPu2LHD0vxPJmatiGzZshEcHKxcS3379lX3\n3r1795g8ebIKrwbHCCCts5Eml4IFC/Lll1+q2q9nzpyhd+/e6cbN+a9U9JkyZWLXrl3JrvrjLszC\nA48CZhImEeHXX3+1WJr7hU+yZcvGt99+69awTk/Spo2j/PLDVkx7WLJmzQo4qm7FxMQA0KtXr1RH\nkqUF+fPnB2DQoEG0bdtWrYwNCAhwSW1ilo80OX36NGvWrPGcoE6sXLmSypUrq+2OHTumq6ilf6Wi\nTw+WVEJMmTLFahESpXTp0oAjfNHqxTNPPvkkI0aMABzW36hRo1SN1ZdeeklZgOkV8wF/4MABFi5c\nSJEiRdR748aNA+CPP/5IkxDgxDDTQgQFBbFgwQIADh065LbzPQxmfp2RI0cycuRISpQoATjqGHfs\n6Ch9kT9/fipUqEBQUBDgCKV96aWXPC6rmXPerF1rphGxep4jLtpHr9FoNBkdq4uOpKbwyMO2Xr16\nSa9evVSxEk+dV7e0beXLl3cpJmK32+XChQty4cIFefrppy2X72HaiBEjVNGRKVOmSO3ataV27dpu\nP2+PHj2kR48eYrPZpFSpUlKqVCnLr8XDtrx580qxYsWkWLFi4u/v7/Hzf/TRR+q7s9vtcuzYMfHx\n8REfHx9PypGswiOPfK6bh8FcmfbFF18A9/2UmkcLb29vlSWwZ8+eXLlyRa1EtboMoybjY078f/HF\nFypE9tChQ/To0cOK+atk5bp5oKI3DGMm8BxwWUTKx+4bD7QC7gIngFdEJDL2vWFAD8AG9BOR9Q8U\nwkOKXqPRaDIYyVL0yfHRfwM0i7MvDCgvIk8Bx4BhAIZhlAM6AU/GHvOVYRiZH0JojUaj0aQxD1T0\nIrIFuBZn348iEhO7uQMwS8CEAAtE5B8ROQX8CVRLQ3k1Go1G85CkRdTNq8Da2NeFgHNO74XH7tNo\nNBqNRaQqjt4wjPeAGOChsyAZhtELsC5vp0aj0fxLSLGiNwzjZRyTtI3k/ozueSDQqVvh2H3xEJFp\nwLTYz9KTsRqNRuMmUqToDcNoBgwG6ouIcw7R74H5hmF8AhQESgLJiTe6AtyK/au5T170NYmLvibx\n0dckYf4N16VIcjo9UNEbhvEd0ADIaxhGODACR5SNNxAWm0N9h4i8LiKHDMNYBBzG4dLpIyIPzDIk\nIgGGYexOTpjQvwl9TeKjr0l89DVJGH1d7vNARS8iLyaw++sk+o8GRqdGKI1Go9GkHTrXjUaj0WRw\n0pOin2a1AOkQfU3io69JfPQ1SRh9XWJJF7luNBqNRuM+0pNFr9FoNBo3YLmiNwyjmWEYRw3D+NMw\njEensrMbMAzjtGEYBwzD+M0wjN2x+/IYhhFmGMbx2L+PWS2nOzEMY6ZhGJcNwzjotC/Ba2A4+Dz2\n3tlvGEYl6yR3H4lck5GGYZyPvVd+MwyjhdN7w2KvyVHDMJpaI7V7MQwj0DCMnwzDOGwYxiHDMPrH\n7v9X3yuJYamij0149iXQHCgHvBibGO3fTEMRedopLGwosFFESgIbY7czMt8QP4leYtegOY61GiVx\nrLKe7CEZPc03xL8mAJ/G3itPi8ga+FclFowBBolIOaAG0Cf2f/+33ysJYrVFXw34U0ROishdYAGO\nxGia+4QAs2NfzwbaWCiL20koiR6JX4MQYI442AHkNgyjgGck9RyJXJPE+FckFhSRv0Rkb+zrm8Af\nOPJq/avvlcSwWtHrJGiuCPCjYRh7YnMBAeQTkb9iX18E8lkjmqUkdg3+7fdP31g3xEwnl96/7poY\nhlEUqAjsRN8rCWK1ote4UkdEKuEYZvYxDKOe85uxOYX+1WFS+hooJgMlgKeBv4D/b++OVRoIgjCO\n/6dQC7XRyjKCb2BhYS2Yzs7KFL6AfZ5BX0CsRKxUTO0b2GhURCSlRdJpKzoWuyFBcpAmDm6+HywX\ncldMhmFgN3d7h7HhxDCzBeACOHD3j+FzqpWB6EY/9iZo08Dd3/KxB1yRptzd/hQzH3txEYapysHU\n1o+7d939y92/gWMGyzNTkxMzmyE1+TN3v8xfq1ZGiG70t8CamdXMbJb0J1IrOKYQZjZvZov9z8AW\n8EjKRyNf1gCuYyIMVZWDFrCX76jYAN6Hpu1F+7W+vEOqFUg52TWzOTOrMf7Ggv+KpU22ToBndz8a\nOqVaGWWcN4hPcgB10usIO0AzOp7APKwC93k89XMBLJPuHngFboCl6FgnnIdz0lLEJ2kddb8qB4CR\n7trqAA/AenT8f5iT0/yb26QmtjJ0fTPn5AXYjo5/QjnZJC3LtIG7POrTXitVQ0/GiogULnrpRkRE\nJkyNXkSkcGr0IiKFU6MXESmcGr2ISOHU6EVECqdGLyJSODV6EZHC/QCu9GT091l2HwAAAABJRU5E\nrkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XFWll5Lseiht",
        "colab_type": "text"
      },
      "source": [
        "**EXERCISE:** Try to understand what the code above is doing. This will help you to better understand your dataset before moving forward. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d9mXAVmRvhrq",
        "colab_type": "text"
      },
      "source": [
        "Let's check the dimensions of a batch."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cNFKWz1GZ4R5",
        "colab_type": "code",
        "outputId": "cc1fd627-b8b0-42d4-d1a7-cd1eeaefc7fb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "source": [
        "for images, labels in trainloader:\n",
        "    print(\"Image batch dimensions:\", images.shape)\n",
        "    print(\"Image label dimensions:\", labels.shape)\n",
        "    break"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Image batch dimensions: torch.Size([32, 1, 28, 28])\n",
            "Image label dimensions: torch.Size([32])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tmaCTw5tXowR",
        "colab_type": "text"
      },
      "source": [
        "## The Model\n",
        "Now using the classical deep learning framework pipeline, let's build the 1 convolutional layer model. \n",
        "\n",
        "Here are a few notes for those who are beginning with PyTorch:\n",
        "- The model below consists of an `__init__()` portion which is where you include the layers and components of the neural network. In our model, we have a convolutional layer denoted by `nn.Conv2d(...)`. We are dealing with an image dataset that is in a grayscale so we only need one channel going in, hence `in_channels=1`. We hope to get a nice representation of this layer, so we use `out_channels=32`. Kernel size is 3, and for the rest of parameters we use the default values which you can find [here](https://pytorch.org/docs/stable/nn.html?highlight=conv2d#conv2d). \n",
        "- We use 2 back to back dense layers or what we refer to as linear transformations to the incoming data. Notice for `d1` I have a dimension which looks like it came out of nowhere. 128 represents the size we want as output and the (`26*26*32`) represents the dimension of the incoming data. If you would like to find out how to calculate those numbers refer to the [PyTorch documentation](https://pytorch.org/docs/stable/nn.html?highlight=linear#conv2d). In short, the convolutional layer transforms the input data into a specific dimension that has to be considered in the linear layer. The same applies for the second linear transformation (`d2`) where the dimension of the output of the previous linear layer was added as `in_features=128`, and `10` is just the size of the output which also corresponds to the number of classes.\n",
        "- After each one of those layers, we also apply an activation function such as `ReLU`. For prediction purposes, we then apply a `softmax` layer to the last transformation and return the output of that.  "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_IYnV4ZBa3cJ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class MyModel(nn.Module):\n",
        "    def __init__(self):\n",
        "        super(MyModel, self).__init__()\n",
        "\n",
        "        # 28x28x1 => 26x26x32\n",
        "        self.conv1 = nn.Conv2d(in_channels=1, out_channels=32, kernel_size=3)\n",
        "        self.d1 = nn.Linear(26 * 26 * 32, 128)\n",
        "        self.d2 = nn.Linear(128, 10)\n",
        "\n",
        "    def forward(self, x):\n",
        "        # 32x1x28x28 => 32x32x26x26\n",
        "        x = self.conv1(x)\n",
        "        x = F.relu(x)\n",
        "\n",
        "        # flatten => 32 x (32*26*26)\n",
        "        x = x.flatten(start_dim = 1)\n",
        "\n",
        "        # 32 x (32*26*26) => 32x128\n",
        "        x = self.d1(x)\n",
        "        x = F.relu(x)\n",
        "\n",
        "        # logits => 32x10\n",
        "        logits = self.d2(x)\n",
        "        out = F.softmax(logits, dim=1)\n",
        "        return out"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "evsFbkq_X6bc",
        "colab_type": "text"
      },
      "source": [
        "As I have done in my previous tutorials, I always encourage to test the model with 1 batch to ensure that the output dimensions are what we expect. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1poxFYqftKov",
        "colab_type": "code",
        "outputId": "0a845d9b-54c8-43b9-c3d6-1abc1b7a4f28",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "source": [
        "## test the model with 1 batch\n",
        "model = MyModel()\n",
        "for images, labels in trainloader:\n",
        "    print(\"batch size:\", images.shape)\n",
        "    out = model(images)\n",
        "    print(out.shape)\n",
        "    break"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "batch size: torch.Size([32, 1, 28, 28])\n",
            "torch.Size([32, 10])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9h_3eZQRHV_P",
        "colab_type": "text"
      },
      "source": [
        "## Training the Model\n",
        "Now we are ready to train the model but before that we are going to setup a loss function, an optimizer and a function to compute accuracy of the model. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3_0Vjq2RHlph",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "learning_rate = 0.001\n",
        "num_epochs = 5\n",
        "\n",
        "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
        "model = MyModel()\n",
        "model = model.to(device)\n",
        "criterion = nn.CrossEntropyLoss()\n",
        "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "44IdrNNeIi_I",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "## compute accuracy\n",
        "def get_accuracy(logit, target, batch_size):\n",
        "    ''' Obtain accuracy for training round '''\n",
        "    corrects = (torch.max(logit, 1)[1].view(target.size()).data == target.data).sum()\n",
        "    accuracy = 100.0 * corrects/batch_size\n",
        "    return accuracy.item()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nK3EcuIOISSR",
        "colab_type": "text"
      },
      "source": [
        "Now it's time for training."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E59hwZlAIVcL",
        "colab_type": "code",
        "outputId": "ab16b14b-8a6e-4568-8500-2f2f5b447a93",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 105
        }
      },
      "source": [
        "for epoch in range(num_epochs):\n",
        "    train_running_loss = 0.0\n",
        "    train_acc = 0.0\n",
        "\n",
        "    model = model.train()\n",
        "\n",
        "    ## training step\n",
        "    for i, (images, labels) in enumerate(trainloader):\n",
        "        \n",
        "        images = images.to(device)\n",
        "        labels = labels.to(device)\n",
        "\n",
        "        ## forward + backprop + loss\n",
        "        logits = model(images)\n",
        "        loss = criterion(logits, labels)\n",
        "        optimizer.zero_grad()\n",
        "        loss.backward()\n",
        "\n",
        "        ## update model params\n",
        "        optimizer.step()\n",
        "\n",
        "        train_running_loss += loss.detach().item()\n",
        "        train_acc += get_accuracy(logits, labels, BATCH_SIZE)\n",
        "    \n",
        "    model.eval()\n",
        "    print('Epoch: %d | Loss: %.4f | Train Accuracy: %.2f' \\\n",
        "          %(epoch, train_running_loss / i, train_acc/i))        "
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch: 0 | Loss: 1.4901 | Train Accuracy: 96.97\n",
            "Epoch: 1 | Loss: 1.4808 | Train Accuracy: 97.90\n",
            "Epoch: 2 | Loss: 1.4767 | Train Accuracy: 98.34\n",
            "Epoch: 3 | Loss: 1.4748 | Train Accuracy: 98.55\n",
            "Epoch: 4 | Loss: 1.4725 | Train Accuracy: 98.81\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QuZxfQc1UIU-",
        "colab_type": "text"
      },
      "source": [
        "We can also compute accuracy on the testing dataset to see how well the model performs on the image classificaiton task. As you can see below, our basic CNN model is performing very well on the MNIST classification task."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YU5WR0BTUHv1",
        "colab_type": "code",
        "outputId": "e0f48883-e06a-4108-a933-0f33b2e56b4f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "test_acc = 0.0\n",
        "for i, (images, labels) in enumerate(testloader, 0):\n",
        "    images = images.to(device)\n",
        "    labels = labels.to(device)\n",
        "    outputs = model(images)\n",
        "    test_acc += get_accuracy(outputs, labels, BATCH_SIZE)\n",
        "        \n",
        "print('Test Accuracy: %.2f'%( test_acc/i))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Test Accuracy: 98.04\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BZz7LAewgGAK",
        "colab_type": "text"
      },
      "source": [
        "**EXERCISE:** As a way to practise, try to include the testing part inside the code where I was outputing the training accuracy, so that you can also keep testing the model on the testing data as you proceed with the training steps. This is useful as sometimes you don't want to wait until your model has completed training to actually test the model with the testing data."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uLQlqGPsVjOB",
        "colab_type": "text"
      },
      "source": [
        "## Final Words\n",
        "That's it for this tutorial! Congratulations! You are now able to implement a basic CNN model in PyTorch for image classification. If you would like, you can further extend the CNN model by adding more convolution layers and max pooling, but as you saw, you don't really need it here as results look good. If you are interested in implementing a similar image classification model using RNNs see the references below. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ztAiTq9HcS_H",
        "colab_type": "text"
      },
      "source": [
        "## References\n",
        "- [Building RNNs is Fun with PyTorch and Google Colab](https://colab.research.google.com/drive/1NVuWLZ0cuXPAtwV4Fs2KZ2MNla0dBUas)\n",
        "- [CNN Basics with PyTorch by Sebastian Raschka](https://github.com/rasbt/deeplearning-models/blob/master/pytorch_ipynb/cnn/cnn-basic.ipynb)\n",
        "- [Tensorflow 2.0 Quickstart for experts](https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/quickstart/advanced.ipynb#scrollTo=DUNzJc4jTj6G) "
      ]
    }
  ]
}